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Abstract 


Recently,  numerous  research  efforts,  most  of  them  concerned  with  superlinear  con¬ 
vergence  of  the  duality  gap  sequence  to  zero  in  the  Kojima-Mizuno-Yoshise  primal-dual 
interior-point  method  for  linear  programming,  have  as  a  primary  assumption  the  con¬ 
vergence  of  the  iteration  sequence.  Yet,  except  for  the  case  of  nondegeneracy  (unique¬ 
ness  of  solution),  the  convergence  of  the  iteration  sequence  has  been  an  important  open 
question  now  for  some  time.  In  this  work  we  demonstrate  that  for  general  problems, 
under  slightly  stronger  assumptions  than  those  needed  for  superlinear  convergence  of 
the  duality  gap  sequence  (except  of  course  the  assumption  that  the  iteration  sequence 
converges),  the  iteration  sequence  converges.  Hence,  we  have  not  only  established 
convergence  of  the  iteration  sequence  for  an  important  class  of  problems,  but  have 
demonstrated  that  the  assumption  that  the  iteration  sequence  converges  is  redundant 
in  many  of  the  above  mentioned  works. 

Keywords:  Linear  programming,  Primal-dual  interior-point  algorithms,  Convergence  of 
iteration  sequence. 

Abbreviated  Title:  Convergence  of  Interior-point  Methods 
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1  Introduction 


This  paper  considers  linear  programs  in  the  standard  form: 

min  cTx 

s.t.  Ax  =  b ,  (1-1) 

x  >  0, 

where  c,  x  €  R",  b  <E  Rm,  A  6  RmXn(ra  <  n)  and  A  has  full  rank  m. 

The  first-order  optimality  conditions  for  (1.1)  can  be  written  as  the  following  2 n  by  2 n 
nonlinear  system  with  non-negativity  constraints  on  the  variables: 

Ax  —  b 

F(x,y)=  By -Be 
^  XYe 

where  B  £  R(n_m)Xn  is  any  matrix  such  that  the  columns  of  BT  form  a  basis  for  the  null 
space  of  A,  X  —  diag(x),  Y  =  diag(y)  and  e  is  the  n-vector  of  all  ones. 

The  feasibility  set  of  problem  (1.2)  is  defined  as 

F  =  {(#,  y) :  x,  y  €  Rn,  Ax  =  b,By  =  Be,  (x,  y)  >  0}. 

A  feasible  pair  ( x,y )  €  T  is  said  to  be  strictly  feasible  if  it  is  positive.  In  this  work  we 
assume  that  strictly  feasible  points  exist.  We  denote  the  solution  set  of  problem  (1.2)  by 

<S  =  {(x*,y*)  :  F(x-,y *)  =  0 ,(s*,y*)  >  0}. 

The  algorithms  under  consideration  in  this  study  are  primal-dual  interior-point  algo¬ 
rithms  as  originally  proposed  by  Kojima,  Mizuno  and  Yoshise  in  [10]  and  are  described  by 
the  following  algorithmic  framework. 

Algorithm  1  (Primal-Dual  Algorithm) 

Given  a  strictly  feasible  pair  (x°,  y°).  For  k  —  0, 1, 2, . . .,  do 

(1)  Choose  <rk  E  [0, 1)  and  set  gk  =  ak^x  ^  y  . 
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(2)  Solve  the  following  system  for  (Axk,Ayk): 


*V.y‘) 


t 


Ax 

Ay 


=  —F(xk,  yk)  +  fik 


'o' 


V  e  / 


(3)  Choose  rk  €  (0, 1)  and  set  ak  =  min(  1,  Tkak)  where 


Ak 

a  — 


-1 


min((Yfc)_1  Axk,  (Ffc)-1Ar/fc) 


(4)  Form  the  new  iterate 


(x‘+1,/+1)  =  (x\yk)  +  a‘(Az‘,  A/). 


(1.3) 


Note  that  the  choice  of  step-length  ak  guarantees  (xk+1  ,yk+1)  >  0.  Moreover,  it  can  be 
easily  verified  that 

(**+*)  V+1  =  (1  -  ak(  1  -  ak))(xk)Tyk  <  (. xk)Tyk .  (1.4) 

Immediately  following  the  presentation  of  the  primal-dual  aglorithm,  one  group  of  re¬ 
searchers  was  mainly  concerned  with  choices  of  the  parameters  in  Algorithm  1  that  led  to 
good  complexity  bounds,  while  another  group  was  concerned  with  choices  of  these  parame¬ 
ters  that  led  to  numerically  effective  algorithms.  Practical  implementations  and  numerical 
experiments  performed  by  MacShane,  Monma,  and  Shanno  [13]  and  by  Lustig,  Marsten,  and 
Shanno  [11],  for  example,  demonstrated  that  primal-dual  algorithms  can  be  computationally 
very  effective. 

The  fast  convergence  of  the  latter  algorithms,  in  part,  motivated  the  recent  flurry  of 
papers  concerned  with  establishing  superlinear  convergence  of  the  duality  gap  to  zero.  For 
linear  programming  applications,  these  papers  include  Zhang,  Tapia,  and  Dennis  [24]  and 
Zhang  and  Tapia  [21].  In  addition,  Ye,  Tapia,  and  Zhang  [20],  McShane  [12],  Ye,  Guler, 
Tapia,  and  Zhang  [19]  and  Mehrotra  [15]  recently  studied  the  Mizuno-Todd-Ye  predictor- 
corrector  algorithm  that  takes  ak  —  1  and  ak  =  0  alternatively.  For  linear  complementarity 
problems,  these  papers  include  Zhang,  Tapia,  and  Potra  [25],  Kojima,  Kurita,  and  Mizuno  [9] 
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and  Ji,  Potra,  Tapia,  and  Zhang  [8].  In  the  context  of  the  present  work,  it  is  important  to 
observe  that  in  establishing  superlinear  convergence  of  the  duality  gap  (or  complementarity) 
to  zero,  all  of  the  above  authors,  except  Ye  et  al  [19],  and  Mehrotra  [15]  assumed  the  con¬ 
vergence  of  the  iteration  sequence.  For  the  Mizuno-Todd-Ye  predictor-corrector  algorithm, 
Ye  et  al  [19]  and  independently  Mehrotra  [15]  obtained  the  impressive  result  of  quadratic 
convergence,  when  counting  the  predictor  and  corrector  steps  as  a  single  step,  without  the 
assumption  of  nondegeneracy  or  the  assumption  of  convergence  of  the  iteration  sequence. 
Whether  such  convergence  can  be  obtained  for  Algorithm  1  without  these  assumptions  is  an 
important  question  and  is  the  subject  of  current  research.  It  is  also  of  concern  to  us  that 
while  in  practice  the  convergence  of  the  iteration  sequence  is  always  observed,  there  is,  to  our 
knowledge,  no  theory  in  the  literature  implying  that  this  should  be  expected  for  problems 
that  do  not  have  unique  solutions.  It  is  the  objective  of  the  present  paper  to  partially  fill 
this  conspicuous  gap. 

This  paper  is  organized  as  follows.  Section  2  contains  notation  and  some  known  results 
that  will  be  used  in  the  development  of  this  paper.  In  Section  3,  we  first  establish  several 
technical  results.  These  technical  results  then  are  used  to  derive  a  bound  on  the  distance  from 
the  (Jc+ l)-th  iterate  to  a  solution  point  in  terms  of  the  distance  from  the  k- th  iterate  to  this 
solution  point  and  the  centering  parameter  a k-  The  parametric  form  of  this  inequality  bears 
a  striking  similarity  to  the  form  of  the  bounded  deterioration  principle  for  secant  updates 
used  by  Broyden,  Dennis  and  More  [1]  to  establish  convergence  for  several  secant  methods. 
For  this  reason,  we  shall  refer  to  the  distance  inequality  established  in  Theorem  3.1  as  the 
bounded  deterioration  of  the  iteration  sequence.  We  then  derive  Theorem  3.2,  a  useful  bound 
on  the  search  steps  of  Algorithm  1,  from  the  bounded  deterioration  of  the  iteration  sequence 
and  the  Hoffman  lemma  [7],  We  present  the  main  theorem  of  this  paper,  Theorem  4.1, 
in  Section  4.  It  states  that,  under  mild  assumptions,  the  iteration  sequence  generated  by 
Algorithm  1  converges.  Finally,  we  make  some  concluding  remarks  in  Section  5. 
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2  Preliminaries 


In  this  section,  we  introduce  notation  and  some  known  results.  For  brevity,  we  use  the 
notation  2  =  ( x,y )  €  R2n,  i.e., 


2;  =  < 


Vi, 


1  <  i  <  n, 
n  <  i  <  2 n. 


(2.1) 


Let  e  =  (0...0  1...  1)T  €  R2n  where  the  numbers  of  zeros  and  ones  are  both  n.  Then 
Algorithm  1  can  be  written  as  the  following  iterative  process: 


2*+‘  =  zt  -  atiF'(zl))-'(F(zt)  -  /e). 


(2.2) 


A  straightforward  calculation  gives 


F'(x,y) 


A  0 
0  B 
Y  X 


(2.3) 


The  following  proposition  is  known;  however  we  could  not  locate  a  simple  proof  for  the 
case  of  linear  programming  in  the  literature  and  therefore,  for  the  sake  of  completeness, 
we  include  a  short  proof.  Observe  that  no  assumptions  are  made  on  the  solution  set  S  of 
Problem  (1.2).  Recall  that  by  the  distance  from  a  point  z  to  a  set  S ,  we  mean 


dist(z,S )  =  inf{||z  —  s||  :  s  £  5}. 

Proposition  2.1  Let  {zk  =  ( xk,yk )}  be  generated  by  Algorithm  1.  Then 

(i)  {zk}  is  bounded. 

(ii)  If  (xk)Tyk  — >  0,  then  dist(zk,S )  — >  0. 

Proof:  (i)  Since  A(xk  —  a:0)  =  0,  B(yk  —  y°)  =  0  and  the  columns  of  BT  form  a  basis  for  the 
null  space  of  A, 


0  =  (xk-  x°)T(yk  -  „°)  =  ( xkfyk  +  (x°)Ty°  -  ( y°)Txk  -  ( x°)Tyk . 
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From  (1.4), 


(y°)Txk  +  ( x°)Tyk  =  (xk)Tyk  +  ( x°)Ty°  <  2 (x°)Ty°. 

The  boundedness  of  ( xk,yk )  now  follows  from  this  inequality  and  the  fact  that  both  (x°,y°) 
and  (xk,yk)  are  positive. 

(ii)  Let  £  denote  the  set  of  limit  points  of  { zk }.  Since  the  duality  gap  converges  to  zero, 
£  C  S.  Given  any  e  >  0,  cover  £  with 

»W=  U  {*:||*-*1<£}. 

z*€£ 

Since  { zk }  is  bounded,  all  but  a  finite  number  of  zk's  are  in  B(e);  otherwise,  {zk}  would  have 
a  new  limit  point  that  is  not  in  £,  and  this  would  be  a  contradiction.  Hence,  there  exists  a 
smallest  number  N  such  that  zk  e  B(e)  for  all  k  >  N.  This  implies  that  dist(zk,S )  <  e  for 
all  k  >  N  and  completes  the  proof.  □ 

Lemma  2.1  below  is  Lemma  2  of  Giiler  and  Ye  [5],  tailored  to  fit  the  needs  of  this  paper. 
Many  interesting  properties  of  interior-point  algorithms  follow  from  this  simple  fact. 

Lemma  2.1  (Giiler- Ye) 

Let  {zk}  —  {{xk,yk)}  be  generated  by  Algorithm  1.  Assume 
Al  (. xk)Tyk  converges  to  zero. 

A2  min (XkYke)/(xk)Tyk  >  7 jn  for  all  k  and  some  7  €  (0, 1). 

Let  z *  be  a  limit  point  of  {zk}.  Then 

(i)  z*  is  a  strictly  complementary  solution  of  Problem  1.2  and 

(ii)  limfc-.oo  inf  zk  >  0  for  every  i  such  that  z*  >  0. 

Recently,  El-Bakry,  Tapia,  and  Zhang  [2]  demonstrated  that  the  relative  interior  of  S 
and  the  set  of  solutions  satisfying  strict  complementarity  coincide.  Furthermore,  the  zero- 
nonzero  structure  of  solutions  in  the  relative  interior  of  S  (equivalently  solutions  satisfying 
strict  complementarity)  is  invariant.  Hence,  under  Assumption  A2  of  Lemma  2.1,  the  limit 
points  of  the  iteration  sequence  generated  by  Algorithm  1  have  considerable  structure. 
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For  any  2*  =  (x*,y*)  in  the  relative  interior  of  S,  define 

1+  =  {i  :  x*  >  0,1  <  i  <  n}  and  /+  =  {*:  y*  >  0, 1  <  i  <  n}.  (2.4) 

Since  the  zero-nonzero  structure  of  the  relative  interior  of  S  is  invariant,  the  above  two  index 
sets  are  independent  of  the  choice  of  2*.  By  strict  complementarity  of  2*, 

=  4+fVv+  =  0-  (2-5) 


3  Bounded  Deterioration 


We  consider  the  following  three  matrices  which  were  introduced  in  Zhang,  Tapia,  and  Den¬ 
nis  [24]: 


P  =  I  -  X12Y-^At(AXY-1At)-1AXW-1^ 

Hp  =  (AY)-*P(*y)-*, 

Hd  =  (IF)-5(/-P)(XF)-i, 

It  is  worth  noting  that  P  and  I  —  P  are  orthogonal  projection  matrices. 
Lemma  3.1  Let  Hv  and  Hd  be  defined  by  (3.2)  and  (3.3),  respectively,  then 

F\x,y)~l 


1 

f°) 

'  XHpw  ^ 

“ ) 

K  YHdw  j 

Proof:  The  proof  follows  from  Proposition  2.1  in  Zhang,  Tapia,  and  Dennis  [24]. 
Lemma  3.2  Let  (x*,y*)  6  S  and  (x,y)  €  T .  Then 

F\z,y)(  X~*'  )  -F(x,y)=  (  ° 

V  y-V  )  \  (X  -  X’)(Y  -  Y')e  ) 

Proof:  The  proof  follows  from  direct  substitution. 


(3.1) 

(3.2) 

(3.3) 


□ 


□ 
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Lemma  3.3  Let  z*  =  ( x*,y *)  (E  S,  z  =  ( x,y )  G  T  and  z+  =  (x+,y+)  be  given  by 


z+  =  z-a[F'(z)]-\F(z)-ye). 


Then 


z+  —  z*  = 


(  XHpW  ^ 

YHdw 


where 


w  =  (X  -  X*)(Y  -  Y*)e  +  (1  -  a)XYe  +  aye. 


Proof:  Using  Lemma  3.2,  we  have 


2+-2*  =  z- z* -a[F'(z)}-\F(z)- ye) 


=  [ F\z) ]  l[F'(z)(z  -  z*)  -  F(z)  +  (1  -  a)F(z)  +  aye] 

=  [H-or1 


/o' 


V  “  ) 


(3.4) 


Now  the  lemma  follows  from  Lemma  3.1. 


□ 


Lemma  3.4  Let  Pi  be  the  i-th  row  (or  column)  of  an  orthogonal  projection  matrix  P.  Then 

\\Pih  <  I- 

Proof:  Since  I  —  P  is  symmetric  and  positive  semi-definite,  Pa  <  1.  Now  P2  =  P  leads  to 

ll-Pilla  =  Pa  <  I-  □ 

We  now  derive  our  bounded  deterioration  relationship  between  the  distances  from  two 
consecutive  iterates,  zk  and  zk+1 ,  to  any  solution  2*. 

Theorem  3.1  Let  {zk  =  (a^,?/*)}  be  generated  by  Algorithm  1  and  let  z*  =  ( x*,y *)  be  any 
solution  to  Problem  1.2.  Assume 

A1  (xk)Tyk  — >•  0. 

A2  min (XkYke)/(xk)Tyk  >  7 /n  for  all  k  and  some  7  G  (0,1). 
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Then  there  exist  constants  ft  >  0  and  ft'  >  0  such  that 


-2,*l  <  (1  +  Z*#)||*‘ 


+  i  =  1,2, 


Proof:  For  simplicity,  we  will  replace  the  superscript  (k+ 1)  by  +  and  drop  the  superscript 
k.  We  let  z  =  z  Az.  Since  z+  —  z  +  olAz  and  a  G  (0, 1],  for  every  index  i  we  have 


zf  £  [min(2,-,  Zi),  max(z;,  £,)]. 


Therefore,  for  z* 

\zt  ~  z*\  <  max(|zt-  -  z* |,  | zi  -  z*\).  (3.6) 

Without  loss  of  generality,  let  us  consider  i  <  n,  i.e.,  z{  =  x{,  z*  =  x*  and  so  on.  The 
proof  for  i  >  n  is  the  same. 

Letting  a.  =  1  in  Lemma  3.3  and  using  the  definition  of  Hp  and  w,  we  obtain 

n 

xi  -  x*  =  Xi  =  xi{si  +  s'"),  (3.7) 

3= 1 

where 

n 

s'i  =  T,(xiyi)~lipij(xjyj)~Hxi  -  x*j)(yj  -  v]), 

3= 1 

and 

s”  =  vY,{xiyi)-*Pij{xjyj)~*— ■ 

j= 1  n 

In  S”  we  used  the  relation  //  =  axTyjn.  Let  us  consider  the  above  two  sums.  Notice  that 


(xj  -  x]){y3  ~y*j)  = 


yj(xj~x*),  if  j  €  /+, 
xj(Vj  -y*)i  if  j€/+. 


We  lose  no  generality  by  assuming  that  (x*,y*)  satisfies  strict  complementarity.  For  if  (3.5) 
holds  for  all  z*  in  the  relative  interior  of  the  solution  set  S,  then  by  continuity  of  the 
absolute  value  and  norm,  it  holds  for  all  points  on  the  relative  boundary  of  S .  As  previously 
mentioned  at  the  end  of  Section  2,  the  points  in  the  relative  interior  of  S  are  exactly  the 
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solutions  satisfying  strict  complementarity.  By  strict  complementarity,  for  each  index  j 
either  x*-  >  0  or  y*  >  0.  Therefore, 


+  £  Pa 

if-i; 


Ik 

Hi  -  y± 

Vj 


Notice  that 

XjVj  _  XjVi^y_  K  n 
Xiyi  xTy  Xiyi  ~  7 

is  bounded  above  and  {xj,  j  £  If}  and  {yj,  j  £  If}  are  bounded  away  from  zero  (see  (ii) 


of  Lemma  2.1).  Moreover,  the  matrix  P  is  an  orthogonal  projection  matrix;  by  Lemma  3.4, 


its  rows  are  bounded  (independent  of  k).  Therefore,  we  can  find  /?,•  >  0  such  that 


i-s?i  <  m*  -  ^ 


In  addition,  by  Assumption  A2  and  Lemma  3.4  again,  we  can  find  j3"  >  0  such  that 


m  =  a 

It  follows  from  (3.7)  that 


'  xTy/n' 
,  XtW  , 


Ep‘> 

j=i 


'  xTy/n 
.  XjVj  , 


<  fto. 


%  ~  *?l  <  *i(|5?l  +  |5?|)  <  \z  -  z*||  +  fiv). 

Finally,  (3.5)  follows  from  (3.6)  and  the  above  inequality.  This  completes  the  proof.  □ 
The  following  inequality,  a  consequence  of  the  bounded  deterioration  and  the  Hoffman 
Lemma  [7],  will  be  a  critical  ingredient  in  the  establishment  of  our  convergence  theory. 

Theorem  3.2  Let  {zk  =  (xk,yk)}  and  {Az*}  be  generated  by  Algorithm  1.  Assume 
Al  ( xk)Tyk  — >  0. 


A2  min (XkYke)/(xk)Tyk  >  7 jn  for  all  k  and  some  7  £  (0, 1). 

Then  there  exist  constants  /?'  >  0  and  (3"  >  0  such  that 

\\Azk\\<f3'(xk)Tyk  +  l3"ak.  (3.8) 

and ,  in  particular, 

\\zk+1~zk\\  <  f3'(xk)Tyk  +  f3"*k.  (3.9) 
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Proof:  Since  zk+1  —  zk  +  akAzk  and  ak  6  (0,1],  it  suffices  to  prove  (3.8).  Notice  that 
the  bounded  deterioration  (3.5)  holds  for  any  ak  £  (0,1].  When  ak  =  1,  from  bounded 
deterioration  (3.5)  we  have  for  any  2*  £  <S 

II  A/||  <  ||(2fc  +  A zk)  -  2*11  +  ||2fc  -  2*||  <  /3|| 2fc  -  2*11  +  /3"ak.  (3.10) 

Now,  consider  the  system  of  linear  inequalities 

Ax  —  b  <  0 
—Ax  —  b  <  0 
By  —  Be  <  0 

-By  +  Bc  <  0  (3.11) 

cTx  —  bT(AAT)~1A(c  — y)  <  0 

—x  <  0 

-y  <  0. 

The  system  (3.11)  is  equivalent  to  the  first-order  conditions  (1.2).  To  see  this  consider  ( x,y ) 
satisfying  (3.11).  We  have  B(y  —  c)  =  0.  Since  both  A  and  B  are  full  rank  and  AT  —  0, 
the  columns  of  AT  form  a  basis  for  the  null  space  of  B.  Hence  there  exists  A  such  that 
y  —  c  =  AT  X.  Therefore 

cTx  —  bT  {AAT)~X  A(c  —  y)  —  cTx  +  bT  A 

=  cTx  +  xTATX  (3.12) 

T 

=  xxy  . 

The  second  line  above  resulted  from  the  substitution  b  =  Ax.  We  have  shown  that  the 
lengthy  expression  in  (3.11)  is  merely  a  linear  inequality  requiring  the  duality  gap  xTy  to  be 
nonpositive.  We  know  that  for  feasible  points  the  duality  gap  is  always  nonnegative.  Hence, 
this  linear  inequality  is  equivalent  to  xTy  —  0.  We  are  now  in  a  position  to  use  the  Hoffman 
Lemma  [7]  with  the  1 2  norm  for  the  choice  of  both  positive  homogeneous  functionals.  This 
lemma  now  says  there  exist  a  constant  f3  >  0  and  wk  €  S  for  every  k  such  that 

||z*  -  •»*!!  <  P(xh)Tyk. 
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Since  2*  €  <5  is  arbitrary  in  (3.10),  it  follows  that  (3.8)  holds  with  /?'  =  (30.  This  proves  the 
theorem.  □ 

Observe  that  when  crk  =  0,  Theorem  3.2  reduces  to  Lemma  3.2  and  Theorem  3.1  of  Ye 
et  al  [19],  i.e.,  a  bound  on  the  Newton  step.  Theorem  3.2  can  also  be  proved  using  this 
bound  on  the  Newton  step  and  the  bound  on  in  the  proof  of  Theorem  3.1,  or  in  perhaps 
other  ways.  We  have  chosen  to  make  Theorem  3.2  a  consequence  of  Theorem  3.1  because 
we  believe  that  Theorem  3.1  is  of  interest  in  its  own  right. 

4  Convergence  Results 

Theorem  4.1  below  represents  our  main  contribution  to  the  convergence  theory  for  the  iter¬ 
ation  sequence  generated  by  primal-dual  interior-point  methods  for  linear  programming. 
Given  any  2*  in  the  relative  interior  of  <S,  let 

I0  =  {i  :  z*  =  0,  1  <  i  <  2n). 

From  the  remarks  above,  IQ  is  invariant  with  respect  to  the  choices  of  2*. 

Recall  that  a  set  in  R71  is  said  to  be  a  continuum  if  it  is  closed  and  cannot  be  written 
as  the  disjoint  union  of  two  proper  closed  subsets.  In  R1  the  continua  are  closed  intervals. 
Clearly,  a  continuum  is  uncountable. 

The  following  conditions  will  be  selectively  used  in  Theorem  4.1.  Their  compatibility  will 
be  demonstrated  in  Proposition  4.1. 

Cl  ( xk)Tyk  ->■  0; 

C2  min (XkYke)/(xk)Tyk  >  7 /n  for  all  k  and  some  7  E  (0, 1). 

C3  ak  -+  0. 

C4  &k  — *  0  at  least  iblinearly. 

C5  <rk  —  0((xk)Tyk). 
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C6  rk  >  t  for  some  r  £  (0, 1). 


Theorem  4.1  Let  { zk  =  ( xk,yk )}  be  generated  by  Algorithm  1.  Then 

(i)  Conditions  C1-C2  imply,  zk  —>  0  for  each  i  £  70. 

(ii)  Conditions  C1-C3  imply, 

•  {zk}  converges  to  some  z*  in  the  relative  interior  of  S,  or 

•  the  limit  points  of  { zk }  form  a  continuum. 

(iii)  Conditions  Cl,  C2,  C4  and  C6  imply,  { zk }  converges  to  some  z*  in  the  relative  interior 
ofS. 

(iv)  Conditions  Cl,  C2,  C5  and  C6  imply,  any  Q  or  R  convergence  behavior  present  in  the 
convergence  of  {{xk)Tyk}  to  zero  is  reflected  as  R-convergence  behavior  in  the  conver¬ 
gence  of  {zk}  to  z*. 

Proof:  (i)  Consider  the  component  sequence  {z^}  for  i  €  I0.  It  is  bounded  from  (i)  of 
Proposition  2.1.  By  Lemma  2.1  and  El-Bakry,  Tapia,  and  Zhang  [2],  all  limit  points  of  {zk} 
are  in  the  relative  interior  of  S  and  have  identical  zero-nonzero  structure;  thus  all  limit  points 
of  {zf}  are  zero.  This  leads  to  zk  — >  0  for  all  i  £  To. 

(ii)  Again  let  C  denote  the  set  of  limit  points  of  {zk}.  Clearly,  C  is  compact.  It  is 
obviously  closed  and  it  is  bounded  since  {zk}  is  bounded  by  (i)  of  Proposition  2.1.  If  C  is 
the  singleton  set  {z*},  then  {zk}  converges  to  2*  since  it  is  a  bounded  sequence  with  only 
one  limit  point. 

Now  suppose  that  £  contains  more  than  one  point  and  is  not  a  continuum.  Then  we  can 
write  £  =  £1  (J  £2  where  £;,*'  =  1, 2,  is  nonempty  and  compact.  By  Hausdorff  separation  we 
can  find  sets  *  =  1,2,  such  that  Oi  is  open,  £;  C  O,  and  the  closures  of  Oi  and  O2  are 
disjoint.  Clearly,  Oi  contains  an  infinite  number  of  zk' s,  or  else,  £t-  would  be  empty;  also  only 
a  finite  number  of  zfc,s  are  not  in  0\  IJ  02.  This  latter  statement  follows  from  the  observation 
that  if  an  infinite  number  of  zfc’s  are  not  contained  in  0\  [_ IO2,  then  they  would  have  a  limit 
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point  not  in  0\  {JO2',  and  therefore  not  in  C.  This  would  be  a  contradiction.  It  follows  that 
we  can  select  a  subsequence  { zk] }  with  the  property  that  zk]  <E  0\  and  zk]+1  6  02-  We 
can  choose  a  thinner  subsequence,  which  we  also  call  {zk>},  with  the  property  that  — »• 

for  some  2*  €  C\  and  zk J+1  €  O2  for  all  j.  However,  this  contradicts  inequality  (3.8)  of 
Theorem  3.2. 

(iii)  It  is  known  (see  Zhang,  Tapia,  and  Dennis  [24],  for  example)  that  C2  and  C6  imply 
ak  >  a  >  0  for  all  k  and  some  a.  From  (1.4)  it  follows  that  the  duality  gap  sequence 
converges  Q-linearly  to  zero.  It  now  follows  from  (3.8)  and  C4  that  there  exist  positive 
constants  rj  and  6  <  1  and  a  positive  integer  K  such  that 

\\zk+1  -  zk\\  <  rj6k,  k  >  K.  (4.1) 

In  a  standard  manner,  (4.1)  implies  that  for  all  positive  integers  N 

|| zk+N  -zk ||  <  ~^—c6k,  k  >  K.  (4.2) 

l—o 

Hence,  {zk}  is  a  Cauchy  sequence  and  is  therefore  convergent. 

(iv)  The  proof  of  (iv)  follows  from  the  derivation  of  the  appropriate  inequality  analogous 

to  (4.2)  and  letting  N  — >  00.  This  proves  the  theorem.  □ 

The  quantities  rk  and  ak  are  under  direct  control  of  the  algorithm  designer.  This  is  an 
extremely  important  point  and  often  missed  by  some  readers.  Even  without  the  knowledge 
of  (iii),  it  seems  that  it  would  be  quite  pathological  for  one  to  choose  ak  —>  0  in  any  manner 
that  did  not  reflect  i?-linear  convergence. 

It  is  worth  pointing  out  the  implications  of  Theorem  4.1  on  the  superlinear  and  quadratic 
convergence  theory  of  Zhang,  Tapia,  and  Dennis  [24].  First,  (iii)  implies  that  the  nondegen¬ 
eracy  assumption  for  quadratic  convergence  is  no  longer  necessary.  Second,  (ii)  implies  that 
the  assumption  of  the  convergence  of  the  iteration  sequence  made  for  superlinear  convergence 
of  the  duality  gap  to  zero  is  no  longer  necessary,  due  to  the  fact  that  in  order  to  establish 
superlinear  convergence  one  only  needs  ||^i+1  —  zk\\  — >  0.  These  facts  have  been  documented 
in  Zhang  and  Tapia  [23]. 
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It  is  perhaps  interesting  to  observe  that  in  the  proof  of  (ii)  by  partitioning  £  so  that  C\ 
is  a  singleton  set  we  have  established  that  either  {zk}  converges  or  £  has  no  isolated  points. 
In  analysis,  a  closed  set  in  which  every  point  is  a  limit  point  of  the  set  is  called  a  perfect 
set.  Clearly,  £  is  a  perfect  set  in  the  case  of  nonconvergence.  It  is  known  that  in  a  separable 
metric  space,  every  perfect  set  must  be  uncountable.  See  Gelbaum  and  Olmsted  [3]  for  a 
discussion  and  see  Hausdorff  [6]  for  a  proof  and  further  discussion.  The  interval  [0, 1]  is  a 
perfect  set  in  R1.  One  might  conjecture  that  a  perfect  set  in  R1  contains  a  continuum,  or  at 
least  has  positive  Lebesgue  measure.  However,  in  R1  the  celebrated  Cantor  set  is  a  perfect 
set  of  Lebesgue  measure  zero;  hence  it  contains  no  continuum.  It  follows  that  this  line  of 
reasoning  would  produce  a  weaker  result  in  (ii). 

It  is  also  interesting  to  note  that  (ii)  of  Theorem  4.1  has  the  flavor  of  a  result  established 
by  Ostrowski  [17]  in  the  context  of  the  gradient  method. 

We  now  demonstrate  the  compatibility  of  Conditions  C1-C6  by  showing  that  choices 
for  ak  and  ak  exist  such  that  these  assumptions  are  simultaneously  satisfied.  This  means 
that  the  algorithm  designer  can  design  an  algorithm  satisfying  C1-C6  or  any  subset  of  these 
conditions. 


Proposition  4.1  For  Algorithm  1  there  exist  parameter  sequences  {crk}  and  {ak} ,  and  the 
corresponding  iteration  sequence  {( xk,yk )}  such  that  Conditions  C1-C6  are  satisfied. 

Proof:  It  suffices  to  demonstrate  that  Conditions  Cl,  C2,  C5  and  C6  are  simultaneously 
satisfied.  Let 

crk  =  mm(cr,p(xk)Tyk)  (4.3) 

where  a  €  (0, 1)  and  p  >  0.  For  each  k  choose  rk  so  that  ak  =  rkak  is  the  largest  number 
in  (0, 1]  such  that 


7  < 


<  T,  i  =  1, 2, . . . ,  n, 


(4.4) 


(xk+1)Tyk+1  jn 

for  some  fixed  constants  7  €  (0, 1/2)  and  T  €  (2 ,n).  It  has  been  shown  (see  Lemma  3.4  of 
Zhang  and  Tapia  [21])  that 


ol  >  min  1, 


<jk(xk)Tyk/ 


n 


2  max  \AxkAyk\ 


(4.5) 
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The  proposition  will  be  proved  if  we  can  show  that  {afc}  is  bounded  away  from  zero. 

When  ak  =  <r,  it  is  known  that  ak  =  fi(^)  (see  [21],  for  example).  Otherwise,  ak  — 
p(xk)Tyk.  By  Theorem  3.2,  \AxkAyk\  =  0((xk)Tyk)2).  Hence  it  follows  from  (4.5)  that  ak 
is  also  bounded  away  from  zero.  This  completes  the  proof.  □ 

5  Concluding  Remarks 

The  main  result  of  this  paper  is  Theorem  4.1.  Among  other  things,  it  says  that  ak  — *  0 
will  guarantee  that  the  step  zk+1  —  zk  converges  to  zero;  but  this  alone  does  not  imply  the 
convergence  of  the  iteration  sequence.  However,  this  convergence  is  guaranteed  if  the  step 
converges  to  zero  fast  enough,  i.e.,  at  least  R- linearly,  and  this  in  turn  is  guaranteed  if  ak 
converges  to  zero  /?- linearly. 

We  now  argue  the  mildness  of  the  assumptions  made  in  Theorem  4.1.  Condition  Cl,  the 
convergence  of  the  duality  gap  sequence  to  zero,  is  necessary  if  we  are  to  consider  conver¬ 
gence  to  a  solution.  Condition  C2  is  a  crucial  ingredient  for  polynomial  convergence  of  the 
duality  gap  sequence  to  zero.  Moreover,  Conditions  C2,  C3  and  rk  —>  1  are  fundamental  in 
establishing  Q-superlinear  convergence  (see  Zhang,  Tapia,  and  Dennis  [24])  of  the  duality 
gap  sequence  to  zero. 

We  now  summarize  the  situation  as  it  concerns  the  convergence  of  the  iteration  sequence 
in  primal-dual  interior-point  methods.  If  the  problem  is  both  primal  and  dual  nondegener¬ 
ate,  then  the  solution  is  unique  and  the  convergence  of  the  iteration  sequence  readily  follows. 
Numerous  papers  in  the  literature  have  assumed  the  convergence  of  the  iteration  sequence 
for  degenerate  problems  in  order  to  facilitate  their  particular  application.  Yet,  prior  to 
the  present  work  there  was  no  theory  whatsoever  addressing  this  issue.  The  present  work 
shows,  under  mild  assumptions,  that  if  the  centering  parameter  sequence  is  chosen  so  that 
{<7fc}  converges  to  zero  at  least  /?- linearly,  then  convergence  of  the  iteration  sequence  results. 
This  result  is  satisfying  because  it  implies  that  the  conditions  guaranteeing  superlinear  con¬ 
vergence  of  the  duality  gap  sequence  to  zero  essentially  imply  convergence  of  the  iteration 
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sequence;  hence  the  assumption  of  iteration  sequence  convergence  is  essentially  redundant 
in  this  theory.  However,  at  this  juncture  we  do  not  know  what  points  in  the  solution  set 
are  acceptable  as  the  limit  points  of  convergent  iteration  sequences.  It  is  our  conjecture 
that  the  iteration  sequence,  under  the  conditions  of  our  theory,  does  not  necessarily  con¬ 
verge  to  the  analytic  center  of  the  solution  set.  Recently,  while  this  paper  was  in  review, 
Zhang  and  Tapia  [22]  demonstrated,  under  mild  assumptions,  that  if  {er*}  is  bounded  away 
from  zero  and  the  iteration  sequence  converges,  then  the  convergence  is  necessarily  to  the 
analytic  center.  The  present  theory  does  not  speak  to  this  case.  Hence,  the  convergence  or 
lack  of  convergence  of  the  iteration  sequence  for  the  case  {&k}  is  bounded  away  from  zero  is 
presently  an  annoying  gap  in  our  understanding.  This  important  question  was  highlighted 
further  when  even  more  recently,  also  while  the  current  paper  was  in  review,  Gonzaga  and 
Tapia  [4]  showed  that  the  iteration  sequence  generated  by  the  Mizuno-Todd-Ye  predictor- 
corrector  primal-dual  interior-point  method  converges,  and  the  convergence  is  to  the  analytic 
center  of  the  solution  set.  The  Mizuno-Todd-Ye  predictor-corrector  method  can  be  viewed 
as  a  special  case  of  the  Kojima-Mizuno-Yoshise  method  (Algorithm  1)  where  the  parame¬ 
ter  sequence  {ent}  is  chosen  so  that  its  terms  alternate  between  the  values  0  and  1.  Hence 
the  algorithm  alternates  between  fidelity  to  the  central  path  and  fidelity  to  the  pure  New¬ 
ton  method.  The  Gonzaga-Tapia  and  the  Zhang-Tapia  theories  do  not  take  away  from  the 
present  theory;  they  do,  however,  add  more  pieces  to  a  very  interesting  and  challenging 
puzzle. 

We  end  this  section,  and  the  paper,  by  briefly  touching  on  what  we  consider  to  to  be  a 
pleasant  turn  of  events.  The  Zhang-Tapia-Dennis  theory  for  fast  convergence  of  the  iteration 
sequence  to  zero  in  the  Kojima-Mizuno-Yoshise  algorithm  was  viewed  by  many  as  having 
the  restrictive  assumption  of  iteration  sequence  convergence.  On  the  other  hand,  a  selling 
point  of  the  Ye-Guler-Tapia-Zhang  theory  of  fast  convergence  of  the  duality  gap  to  zero 
in  the  Mizuno-Todd-Ye  algorithm  was  that  it  did  not  make  the  assumption  of  iteration 
sequence  convergence.  The  present  theory  shows  that  the  assumption  of  iteration  sequence 
convergence  is  not  restrictive  in  the  Zhang-Tapia-Dennis  theory,  and  the  recent  Gonzaga- 
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Tapia  theory  shows  that  the  Ye-Giiler-Tapia-Zhang  theory  is  not  more  general  than  that 
which  would  result  from  adding  the  assumption  of  iteration  sequence  convergence.  It  seems 
then  that  fast  convergence  of  the  duality  gap  sequence  to  zero  and  convergence  of  the  iteration 
sequence  go  hand  in  hand. 
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